{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<h1> Create TensorFlow model </h1>\n",
    "\n",
    "This notebook illustrates:\n",
    "<ol>\n",
    "<li> Creating a model using the high-level Estimator API \n",
    "</ol>"
   ]
  },
  {
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
 "!sudo chown -R jupyter:jupyter /home/jupyter/training-data-analyst"
   ]
  },
  {

"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
 "# Ensure the right version of Tensorflow is installed.\n",
 "!pip freeze | grep tensorflow==2.1"
 ]
},
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# change these to try this notebook out\n",
    "BUCKET = 'cloud-training-demos-ml'\n",
    "PROJECT = 'cloud-training-demos'\n",
    "REGION = 'us-central1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['BUCKET'] = BUCKET\n",
    "os.environ['PROJECT'] = PROJECT\n",
    "os.environ['REGION'] = REGION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "if ! gcloud storage ls | grep -q gs://${BUCKET}/; then\n",
    "  gcloud storage buckets create --location=${REGION} gs://${BUCKET}\n",
    "fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<h2> Create TensorFlow model using TensorFlow's Estimator API </h2>\n",
    "<p>\n",
    "First, write an input_fn to read the data.\n",
    "<p>\n",
    "\n",
    "## Lab Task 1\n",
    "Verify that the headers match your CSV output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import shutil\n",
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Determine CSV, label, and key columns\n",
    "CSV_COLUMNS = 'weight_pounds,is_male,mother_age,plurality,gestation_weeks,key'.split(',')\n",
    "LABEL_COLUMN = 'weight_pounds'\n",
    "KEY_COLUMN = 'key'\n",
    "\n",
    "# Set default values for each CSV column\n",
    "DEFAULTS = [[0.0], ['null'], [0.0], ['null'], [0.0], ['nokey']]\n",
    "TRAIN_STEPS = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Lab Task 2\n",
    "\n",
    "Fill out the details of the input function below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Create an input function reading a file using the Dataset API\n",
    "# Then provide the results to the Estimator API\n",
    "def read_dataset(filename_pattern, mode, batch_size = 512):\n",
    "  def _input_fn():\n",
    "    def decode_csv(line_of_text):\n",
    "      # TODO #1: Use tf.decode_csv to parse the provided line\n",
    "      # TODO #2: Make a Python dict.  The keys are the column names, the values are from the parsed data\n",
    "      # TODO #3: Return a tuple of features, label where features is a Python dict and label a float\n",
    "      return features, label\n",
    "    \n",
    "    # TODO #4: Use tf.gfile.Glob to create list of files that match pattern\n",
    "    file_list = None\n",
    "\n",
    "    # Create dataset from file list\n",
    "    dataset = (tf.compat.v1.data.TextLineDataset(file_list)  # Read text file\n",
    "                 .map(decode_csv))  # Transform each elem by applying decode_csv fn\n",
    "    \n",
    "    # TODO #5: In training mode, shuffle the dataset and repeat indefinitely\n",
    "    #                (Look at the API for tf.data.dataset shuffle)\n",
    "    #          The mode input variable will be tf.estimator.ModeKeys.TRAIN if in training mode\n",
    "    #          Tell the dataset to provide data in batches of batch_size \n",
    "\n",
    "    \n",
    "    # This will now return batches of features, label\n",
    "    return dataset\n",
    "  return _input_fn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Lab Task 3\n",
    "\n",
    "Use the TensorFlow feature column API to define appropriate feature columns for your raw features that come from the CSV.\n",
    "\n",
    "<b> Bonus: </b> Separate your columns into wide columns (categorical, discrete, etc.) and deep columns (numeric, embedding, etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Define feature columns\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Lab Task 4\n",
    "\n",
    "To predict with the TensorFlow model, we also need a serving input function (we'll use this in a later lab). We will want all the inputs from our user.\n",
    "\n",
    "Verify and change the column names and types here as appropriate. These should match your CSV_COLUMNS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Create serving input function to be able to serve predictions later using provided inputs\n",
    "def serving_input_fn():\n",
    "    feature_placeholders = {\n",
    "        'is_male': tf.compat.v1.placeholder(tf.string, [None]),\n",
    "        'mother_age': tf.compat.v1.placeholder(tf.float32, [None]),\n",
    "        'plurality': tf.compat.v1.placeholder(tf.string, [None]),\n",
    "        'gestation_weeks': tf.compat.v1.placeholder(tf.float32, [None])\n",
    "    }\n",
    "    features = {\n",
    "        key: tf.expand_dims(tensor, -1)\n",
    "        for key, tensor in feature_placeholders.items()\n",
    "    }\n",
    "    return tf.compat.v1.estimator.export.ServingInputReceiver(features, feature_placeholders)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Lab Task 5\n",
    "\n",
    "Complete the TODOs in this code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Create estimator to train and evaluate\n",
    "def train_and_evaluate(output_dir):\n",
    "  EVAL_INTERVAL = 300\n",
    "  run_config = tf.estimator.RunConfig(save_checkpoints_secs = EVAL_INTERVAL,\n",
    "                                      keep_checkpoint_max = 3)\n",
    "  # TODO #1: Create your estimator\n",
    "  estimator = None\n",
    "  train_spec = tf.estimator.TrainSpec(\n",
    "                       # TODO #2: Call read_dataset passing in the training CSV file and the appropriate mode\n",
    "                       input_fn = None,\n",
    "                       max_steps = TRAIN_STEPS)\n",
    "  exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)\n",
    "  eval_spec = tf.estimator.EvalSpec(\n",
    "                       # TODO #3: Call read_dataset passing in the evaluation CSV file and the appropriate mode\n",
    "                       input_fn = None,\n",
    "                       steps = None,\n",
    "                       start_delay_secs = 60, # start evaluating after N seconds\n",
    "                       throttle_secs = EVAL_INTERVAL,  # evaluate every N seconds\n",
    "                       exporters = exporter)\n",
    "  tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Finally, train!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Run the model\n",
    "shutil.rmtree('babyweight_trained', ignore_errors = True) # start fresh each time\n",
    "tf.compat.v1.summary.FileWriterCache.clear()\n",
    "train_and_evaluate('babyweight_trained')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The exporter directory contains the final model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Copyright 2020 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
